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Abstract. We generalize a formula obtained independently by A. Reifegerste 
and J. Sjostrand for the sign of a permutation under the classical Robinson- 
Schensted map to a family of domino Robinson-Schenstod algorithms. 

1. Introduction 

In their work verifying R. Stanley's sign imbalance formula, A. Reifegerste and 
J. Sjostrand independently obtained the following remarkable formula for reading 
the sign of a permutation from its image under the classical Robinson-Schensted 
map. It is based on two tableaux statistics e and sign: 

Theorem ([IH]j[in])- Consider w E Sn and let RS{w) = {P,Q) be its image under 
the classical Robinson-Schensted map. Then 

sign{w) ~ (—1)"^ • sign(P) ■ sign{Q). 

In f6^, D. Garfinkle, building on the work of D. Barbasch and D. Vogan in [P, in- 
troduced a generalization of the Robinson-Schensted algorithm relating elements of 
the other classical Weyl groups and same-shape pairs of domino tableaux. This al- 
gorithm was further extended to a one-parameter family of maps Gr by M. A. A. van 
Leeuwen using domino tableaux with non-empty core. For large values of r, van 
Leeuwen's maps recover yet another generalization of the Robinson-Schensted maps 
in this setting introduced by R. Stanley [HJ §6] . The aim of this paper is to ad- 
dress the natural question of whether it is again possible to recover the parity of 
the length function, which we call its sign, from the tableaux images of these maps. 

In fact, this has already been done for the Stanley map in [15' in the more general 
context of complex reflection groups. For the family of maps Gr, our main result 
relies on three domino tableaux statistics d, spin, and sign defined in Section [2.3| 
Let Hn be the Weyl group of type i?„. 

Theorem. Consider w € i?„, and let Gr{w) — {P,Q) be its image among same- 
shape standard domino tableaux of rank r. Then 

sign{w) = {-lY ■ . sign{P) ■ sign[Q) 

The proof involves three steps. We first verify the equation for large r by trans- 
lating between Stanley's map and Gr and appealing to the formula established in 
[15] . It is then possible to extend the result to involutions in Hn for arbitrary r 
using a relation between consecutive maps Gr described in [17) , and finally to to all 
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signed permutations by tracking the behavior of the estabhshed sign formula under 
M. Ta§km's plactic relations introduced in [53]. 

The domino tableaux Robinson-Schensted algorithms appear in the work classi- 
fying Kazhdan-Lusztig cells in unequal parameter Iwahori-Hecke algebras of type 
B, see [3]. At least conjecturally, for certain values of the parameter, one-sided 
cells correspond to plactic and coplactic classes for the maps Gr- As corollary to 
the above sign formula, we note that the Mobius function for the Bruhat order, 
ubiquitous in Kazhdan-Lusztig theory, is well-behaved with respect to these cells. 
First described by by D.-N. Verma in [21], the Mobius function takes the form 

where £ is the length function on the Weyl group. In type B, the values of can be 
readily read off from the tableaux of v and w arising from the maps Gr- Further, 

Corollary. Consider x,x',y,y' G Hn and fix a map Gr- Suppose that the left 
tableaux of the pair x and y as well as x' and y' are the same, and the right 
tableaux of the pairs x, x' and y, y' similarly agree- Then 

2. Preliminaries 

We define the notions of standard and domino tableaux, describe a family of 
Robinson-Schensted maps, and detail several tableaux statistics which will be nec- 
essary for our work. 

2.1. Partitions and tableaux. Our first objective is to define the notions of 
standard Young, bi-, and domino tableaux. A non- increasing sequence of positive 
integers A — (Ai, A2, . . . , At) is called a partition of the integer n = J^i'^i- We 
will write A h n and |A| — n- Partition notation can often be abbreviated by 
using exponents to denote multiplicity; for instance, (4,4,3,3,3,1) can be written 
as (4^,3^,1). We will identify a partition with its Young diagram [A], or a left- 
justified array of squares containing A^ squares in row i. 




1 1 1 I 



(a) (b) (c) 

Figure 1. (a) The Young diagram of the partition (4,3^, 1), (b) 
a standard bitableau of shape ((3, 2, 1), (2, 1)), and (c) a standard 
domino tableau of rank 2. 

If the rightmost square s of a row in a Young diagram [A] can be removed while 
leaving another Young diagram [AJ^, it will be called a corner of [A]. Beginning 
with [A], one can start successively removing corners until this process inevitably 
terminates after n steps with the empty partition. If the square removed at the ith 
step of this procedure is labeled with the number n — i + 1, the result is called a 
standard Young tableau- For a tableau T, we will write sh(T) for the underlying 
partition, |r| for \sh[T)\, SYT{X) for the set of all standard Young tableaux of 
shape A, and SYT{n) for the set of standard Young tableaux with n boxes. 
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We will call an ordered pair of partitions (A, /i) a bipartition of n if |A| + = n. 
A square of ([A], [/i]) is a corner if it is a corner of either [A] or [fi]. Successive 
removal of corners starting with ([A], [fi]) terminates after n steps, and if the square 
removed at the ith step is labeled with n — i + 1, the result will be called a standard 
bitableau. The shape of a bitableau is the pair of its underlying partitions; we will 
write SBT{\,ii) for the set of all standard bitableaux of shape (A,/i), and SBT{n) 
for the set of standard bitableaux with n boxes. 

Two squares of a Young diagram are adjacent if they share a common side. 
Adjacent squares s, t in [A] form a domino corner if s is a corner for [A] and t is 
a corner for [\]s- Beginning with [A], one can start successively removing domino 
corners and continue until this is no longer possible, say after n steps. The resulting 
shape is a staircase partition 5^. = (r, r — 1, r — 2, . . . 1) for some r > and is 
independent of the order of removal of domino corners, see [14^ . The partition 5^ 
is known as the 2- core of A. If the squares of [A] corresponding to the 2-core are 
labeled with and the domino removed at the ith step is labeled with n — i + 1, 
the result is a standard domino tableau of rank r. The set of all standard domino 
tableaux of shape A with 2-core 5r will be denoted by SDTr{X) while SDTr{n) will 
denote the set of all standard domino tableaux consisting of the 2-core 6r and n 
dominos. We will call the set of squares in a domino tableau T labeled with the 
core of T. 

2.2. Robinson-Schensted maps. Consider a permutation ui e S'n. We will write 
it in one- line notation as wiW2 ■ ■ - Wn with each entry Wi S N„. The classical 
Robinson-Schensted map establishes a bijection between permutations in Sn and 
same-shape pairs of standard Young tableaux in SYT(n) x SYT{n) via an insertion 
and a recording algorithm. We assume the reader is familiar with the basics; details 
can be found in [S] or [55]. We will write RS{w) = {P{w),Q{w)) for the image of 
a permutation under this map. 

A signed permutation is a permutation together with a choice of sign for each 
of its entries. We will again use one-line notation, using a bar over a letter to 
denote the choice of a negative sign. The set of signed permutations on n letters 
forms a group under composition and multiplication of signs; it is isomorphic to 
the hyperoctahedral group Hn — 'L2lSn and is generated by 

Si = \ 2 . . . i + \ i . . .n, and t — 1 2 ... n 

for I < i < n. Let £ be the length function on i/„ defined in terms of this generating 
set and write sign{w) = (— 1)^^"'^. 

We are interested in two generalizations of the Robinson-Schensted map in this 
setting. The first establishes a map 

Goo : H,, — > SBT{n) x SBT{n) 

that is a bijection onto same-shape pairs of bitableaux. Given a signed permutation, 
the insertion bitableau for w S iJ„ is constructed by a variant of the classical 
insertion algorithm. Positive letters are inserted into the first tableau and negative 
into the second following their order of appearance in the one line notation for w. 
The recording bitableau tracks the shape of the insertion bitableau at each step. 
See EI]- 

Example 2.1. Consider the signed permutation w — (4 321) e H^^. The sequence of 
bitableaux constructed by successive insertion of the letters of w is: 



4 



THOMAS PIETRAHO 



Keeping track of the shapes appearing in this sequence, we can construct another 
bitableau of the same shape obtaining: 

°"<"')-((SJ).(IJ 

The second generalization of the classical Robinson-Schensted algorithm to the 
hyperoctahedral groups map has image within same-shape pairs of domino tableaux. 
In fact, for every non-negative integer r, there is a map 



Gr ■ H„ 



SDTr{n) X SDTr{n) 



that is a bijection onto same-shape pair of domino tableaux of rank r. Starting with 
the diagram [5r\ , a tableau is constructed via a domino insertion procedure inspired 
by the classical algorithm. Positive letters are inserted as horizontal dominos in the 
first row of the tableau while negative ones are inserted as vertical dominos in its 
first column. As long as the two types of dominos do not interact, the procedure 
is very similar to classical insertion; when they do, a more complicated bumping 
procedure becomes necessary. 

Example 2.2. Let r = 2 and w = (4 321) S H^. The sequence of domino tableaux 
constructed by successive insertion of the letters of w into the 2-core [^2] is: 



w 
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Keeping track of the shapes appearing in this sequence, we can construct another 
domino tableau of the same shape obtaining: 
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An initial version of the hyperoctahedral Robinson-Schensted maps Gq and Gi 
is due to D. Barbasch and D. Vogan [1], but was only described later in terms of 
domino insertion by D. Garfinkle [B]. Van Leeuwen showed that the bijection holds 
for all r and described the map using growth diagrams For a more formal 

description of the Gr where all the details of the insertion and bumping procedures 
may be found, see jS] or [T^ . 

When r is sufficiently large relative to n, inserted dominos corresponding to the 
positive and negative letters of w do not interact and it is easy to see that it is 
possible to recover Goo^w) from Gr(w). Thus in this sense, Goo is an asymptotic 
version of the Gr- 

2.3. Tableaux statistics. Our ultimate goal is to be able to read off the sign 
character of a signed permutation from its image under the various Robinson- 
Schensted maps. To do so, we first need to define and extend a few tableaux 
statistics. 

Definition 2.3. Let T E SYT{n). A pair of entries is an inversion in T if 
j < i and j is contained in a row strictly below the row of i. 
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Figure 2. Images oi w — (4321) under the domino Robinson- 
Schensted maps Gr 



We first extend the definitions of inversions and sign to bitableaux as well as 
domino tableaux. 

Definition 2.4. Let T = {Ti,T2) E SBT{n). A pair of entries (i, j) is an inversion 
in T if it is either an inversion in the standard Young tableaux Ti or T2, or j < i 
and j is contained in Ti while i is contained in T2. We define spin(T) = IT2I/2. 

Definition 2.5. Let T e SDT{n). We will say that a square of T is marked if the 
sum of its coordinates is even. A pair of positive entries (i, j) is an inversion in T if 
j < i and j labels a marked square in a row strictly below the marked square with 
label i. 

For a standard Young, bi-, or domino tableau T, we will write Inv(T) for the 
set of its inversions and inv{T) for the the cardinality of this set. The sign of the 
tableau T will then be defined as sign{T) — (— 1)™^(^). Note that when applied to 
domino tableaux, the present notion of sign differs in general from the traditional 
one as defined in |25) . for example. In particular, using the present definition, the 
sign of a domino tableau depends on more than just the underlying domino tiling 
of the domino tableau shape, see 25, Prop. 9]. 



Example 2.6. Consider the following three tableaux: 

S = 



T = 
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According to the above definitions, their sets of inversions are Inv{T) = {(4, 3), (6, 5)} 
Inv{S) = {(2, 1), (3, 1), (4, 1), (7, 1), (8, 1), (6, 4), (6, 5), (7, 5), (8, 5)}, and InviU) = 
0. 

We also define a few statistics special to domino tableaux. Let v{T) be the 
number of vertical dominos in T and let spin{T) — v{T)/2. For a signed per- 
mutation w, the total color tc{w) is the number of negative letters in its one line 
notation. This statistic is particularly well-behaved with respect to the domino 
Robinson-Schensted maps Gr- 

Theorem 2.7 (|19|. |10|). Consider a signed permutation w € iJ„ and further let 
Gr{w) = (PjQ) be its image under the domino Robinson-Schensted map Gr among 
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same-shape pairs of domino tableaux with 2-core Sr- Then 

tc{w) = spin{P) + spin{Q). 

This result is clear when r > n — 1 . It was verified for r = in |19) and extended to 
all r in [10] . Of particular interest to us is the immediate observation that the sum 
of the spins of the recording and tracking tableaux for the maps Gr is independent 
of r. 

For a standard domino tableau T, let eh{T) and ev{T) denote the number of 
horizontal dominos in even index rows and vertical dominos is even index columns 
of T, respectively. If we let d{T) denote the number of squares with a positive label 
which lie both in an even row and even column of T, then eh{T) + ev{T) = d{T). 

2.4. The type A sign character. We can now state the result of A. Reifegerste 
and J. Sjostrand which our main results generalize. 

Theorem 2.8 ([IH],E3)- Consider a permutation w G Sn and let RS{w) — {P, Q) 
be its image under the classical Robinson-Schensted map. Then 

sign{w) = (— 1)"^ • sign{P) ■ sign{Q) 

where e — e(P) is the sum of the lengths of all the even-index rows of P. 

The following is a special case of a result on the sign characters of the complex 
reflection groups G{r,p,n). 

Theorem 2.9 ([H]). Consider w € i?„ and let Goo('w) = {P,Q) be its image 
among same-shape standard bitableaux, where P = {Pi,P2) and Q ~ {Qi,Q2)- 
Then 

sign{w) = {-ly ■ (^iypMP)+'^pMQ} . sign{P) ■ sign{Q) 

where e = e(Pi) + e{P2) is the sum of the lengths of all the even-index rows of the 
constituent tableaux of P. 

2.5. Cycles. The most technical aspect of this work lies in the notion of a cycle 
in a domino tableau. First defined in [5], cycles have appeared in various settings, 
including [J] , [T^] , [13] , and [T7] . We provide a brief introduction, beginning with 
a few definitions. 

For a standard domino tableau T G SDTr{n), we will say the square Sij in row 
i and column j of T is variable when z + j = r mod 2, otherwise, we will call it 
fixed. Following [B] , we further differentiate variable squares by saying Sij is of type 
X if i is odd and of type W otherwise. Write D{k, T) for the domino labeled by the 
positive integer fc in T and supp D{k,T) for its underlying squares. Write label Sij 
for the integer label of Sij in T and let label Sij = if either i or j is less than or 
equal to zero, and label Sij — oo ii i and j are positive but Sij is not a square in T . 

Definition 2.10. Let suppD{k, T) = {s^, s^+i .,} or {s^ s.y} and suppose that 
the square Sij is fixed. Define a new domino D' [k) labeled by the integer k by letting 
supp D' (k,T) equal to 

(1) {sij,Si_ij} if fc < label Si-ij+i 

(2) {sij,Sij+i} if fc > label Si^i^j+i 

If supp D{k, T) = {sij, Si^i j} or {s^ j+i, Sij} and the square Sij is fixed, then define 
suppD'{k,T) to be 
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(1) {sij,Sij-i} if fc < label Si+i^j^i 

(2) {sij,Si+ij} if fc > label Si+ij_i 

Definition 2.11. For T G SDTr{n), the cycle c = c{k,T) through k is the set of 
integers defined by the condition that Z e c if either 

(1) l = k, 

(2) supp D{l,T) n supp D' {m,T) ^ for some m G c, or 

(3) suppD'(l,T) n supp D{'m,T) ^ for some m G c. 

We identify the labels contained in a cycle with their underlying dominos. 

If c is a cycle in T, it is possible to construct a tableau MT{T, c) by replacing 
every domino -D(^,T) S c by the shifted domino D'{l,T) defined above. This map 
produces a standard domino tableau, preserves the labels of the fixed squares of T, 
and changes the labels of the variable squares in c. The shape of MT{T, c) either 
equals the original shape of T, or one square will be removed (or added to the core) 
and one will be added. In the first case, the cycle c is called closed; otherwise, 
it is called open. For an open cycle c of a tableau T, we will write S'h(c) for the 
square that has been removed (or added to the core) by moving through c. It is 
the beginning square of the c. Similarly, we will write Sf{c) for the square that is 
added to the shape of T, the final square of c. Note that Sb{c) and Sf{c) are always 
variable squares. 

Example 2.12. Below are two diagrams of a standard domino tableau of rank 2, the 
first unadorned, and then with its cycles highlighted and the final squares of the 
open cycles displayed as dashed boxes. There is one closed cycle, c = {11, 12}. 

| 0| j I 2~| | | 0| 1 I "2~f i 

5 I 6 I 5 I 6 i ! 
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Each square adjacent to the core in T G SDTj.{n) is the beginning square for some 
open cycle. We call the set of all such open cycles A(T). One fact about moving 
through open cycles will be especially relevant and is apparent in the example 
above. 

Proposition 2.13 ([S],[I1]). Consider T G SDTr{n). If c G A(T), then the vari- 
able squares Sb{c) and Sf{c) are both of type X or both of type W . In particular, 
Sb{c) lies in an even row and column of T iff Sf{c) does as well. 

The order in which one applies the moving through map to cycles in a set U is 
immaterial by [5](1.5.29), allowing us to write MT{T, U) for the tableau obtained 
by moving through all of the cycles in the set U . 

3. Sign of colored permutations 

Based on the definitions of domino tableaux statistics enumerated in Section [2.31 
we are ready to state a formula for reading the sign of a colored permutation from 
its image under any of the domino Robinson-Schensted maps. 



8 



THOMAS PIETRAHO 



Theorem 3.1. Consider a signed permutation w G Hn and let Gr{w) = (P, Q) be 
its image among same- shape standard domino tableaux of rank r. Then 
sign{w) ^ {-lY ■ {^-iypMP)+spin(Q) . ^j^^^p) . sign{Q) 

where d — d{P) denotes the number of non-core squares which lie concurrently in 
an even row and column of P , and spin denotes the spin of a tableau. 

Our first goal is to verify the theorem for involutions in _ff„. There are two main 
tools, the sign formula for colored permutations under Goo derived from |15j and 
a description of the relationship between the maps Gr and Gr+i obtained in [T7] . 
When r is large relative to n, the relationship between Goo and Gr is simple and it 
is a trivial task to translate one sign formula into the other. Using the map of [T7], 
we then extend the result on involutions to all r. 

Under the maps Gr, the left and right tableaux for an involution in if„ coincide, 
see [?]. To complete our proof, we examine the behavior of the sign character 
under the plactic relations on obtained in [23 • As plactic relations generate 
the equivalence classes of having the same left tableau under Gr, this extends the 
theorem to all of i/„ . 

3.1. Involutions. The goal of this section is to verify the claimed sign formula for 
involutions in Hn- We follow the approach outlined above and start by translating 
the formula for the asymptotic map Goo to the maps G,. for r > n — 1. 

Lemma 3.2. Let i G Hn be an involution and write Gr{i) — {P, P)- For r > n~\, 

sign{i) = {-lY ■ (-1)2''P™(-P) 

where d — d{P) denotes the number of non-core squares which lie concurrently in 
an even row and column of P, and spin denotes the spin of a tableau. 



Proof Let Goo(i) = (R^R). By Theorem [2J| if we write R = (i?i,i?2), then 
sign{i) = (— 1)'^+I^2| where e = e(i?i) + e(i?2) is the sum of the lengths of all the 
even-index rows of the constituent tableaux of R. Now note that tc{i) = |i?2| = 
2spin{P), so it remains to show that d = e. The squares in even-indexed rows of 
i?i correspond to horizontal dominos in even rows of P and the squares in even- 
indexed rows of i?2 correspond to vertical dominos in even columns of P. Equality 
follows. □ 

Next, we extend this formula to all values of r. Let r and r' be non-negative 
integers and suppose that T G SDTr{n). Following [T^, we define a map 

try : SDTr{n) — > SDTr'{n) 

by setting tr.r'{T) = T' whenever G-\T,T) = G-,^{T',T'). When r and r' are 
consecutive integers, this map has a particularly simple description in terms of 
cycles in a domino tableau. Given T G SDTr{n), it is easy to produce a domino 
tableau of rank r -I- 1 by moving through open cycles; simply move thorough all 
the open cycles in A(r). It is clear that the 2-core of the resulting tableau is <5r+i- 
What is perhaps surprising is that this map coincides with tr^r+i- 

Theorem 3.3 ([H]). Let t^.r+i : SDTr{n) SDTr+i{n) be defined as above. 
Then 

tr,r+l{T)^MT{T,A{T)). 
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Example 3.4. Consider the involution i — (59711 16310284) e Hn. Its image 
under the domino Robinson-Schensted algorithm G2 is a pair of tableaux {P, P) 
with P as below. 



P 
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There are three open cycles in A(P), mainly {1,2}, {3,4}, and {5,6}. Moving 
through all three produces the tableau P'. As claimed by Theorem |3 .31 this is also 
the image of i under G3; mainly 6*3(1) = (P', P'). 

In order to extend the involution sign formula to all values of r, it suffices to 
check that our tableau statistics are well-behaved with respect to the maps tr^r+i 
for all values of r. 

Lemma 3.5. Let i G Hn be an involution and write Gr{i) = {P,P). For r >0, 

sign{i) = {-lY ■ {-if^P^^(P) 

where d = d{P) denotes the number of non-core squares which lie concurrently in 
an even row and column of P, and spin denotes the spin of a tableau. 

Proof Let P' = tr,r+i{P)- We verify that d{P) = d(P') and spin(P) = spin{P'), 
showing that the right hand side of the claimed equation is independent of r. Since 
the theorem holds for large r by Lemma [3]2] the result will follow. 

First, note that spin{P) = spin{P') since both equal tc{i) by Theorem 2.7 
Since P' = MT{P, A(P)) by Theorem 3.3 the difference between the shapes of the 
two tableaux are the beginning and final squares for the open cycles in A(P). In 
this process, d{P) is reduced by one for each cycle whose beginning square lies in 
an even row and an even column. It increases by one for each cycle whose final 
square lies in an even row and an even column. But by Proposition |2. 13"! cycles in 
A(P) whose final square lies in an even row and column are precisely those whose 
beginning square lies in an even row and an even column. Thus d{P) — d{P'). □ 



3.2. Extension to In this section we complete the proof of Theorem |3.1| 
Each of the Robinson-Schensted algorithms Gr suggests a equivalence relation on 
Hn, with two colored permutations equivalent if and only if they share same left 
tableau in the image of Gr- While this family of relations has significance in repre- 
sentation theory and the Kazhdan-Lusztig theory of cells, see [5] and [3], we have 
an opportunity to use it toward our more modest purpose. In |23| . M. Ta§kin de- 
scribed a set of generators for each of the above equivalence relations. To prove 
Theorem |3.H we track the action of each generator on left tableaux as well as the 
sign of the corresponding colored permutation. 

We reproduce the definitions of five operators on Hn originally appearing in |23| . 
The first is derived from the original Knuth relations of [9 . Precursors to the next 
two appear in [2] and [T as plactic relations for Goo and Gq. The final two are 
designed to deal with two specific situations appearing among domino tableaux, 
especially of higher rank. Write w — W1W2 ■ ■ ■ Wn for a colored permutation and 
adopt the convention that z = z. 
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1. If Wi < Wi+2 < or Wi < Wi-i < Wi+i for some i < n, then 



Dl{w) =Wi... Wi-i{Wi+iWi)Wi+2 ...Wn- 

2. If r > and if there exists < i < r such that un and Wi+i have 
opposite signs, then 



D'^{w) =Wi... Wi-i{Wi+iWi)Wi+2 ...w, 



3. Suppose that \w\\ > for all 1 < i < r + 2 and ^2 • ■ • Wr+2 
is obtained by concatenating some positive decreasing sequence to 
the end of some negative increasing sequence (or vice versa), where 
at least one of the sequences is nonempty. Then 



D^{'w) = W1W2 ■ ■ - w. 



4. Let k> 1 such that t = (fc + l)(r + A; + 1) < m and suppose 

W = €i... €k€k+l Z Wt+l ...Wn 

where each €i is a sequence of the form £j = aj ^+r • • • O'i.ibi.i ■ ■ ■ &j,i 
for 1 < i < k and £/c+i = ak+i,k+r ■ ■ ■ aft+i.i- Further suppose that 
the integers atj and bij, whenever they appear among wi . . . Wt-i = 
£1 . . . satisfy the following conditions: 
Uij > and bij < (or vice versa) 

< < ki+i,j| and \biji\ < \bij\ < \bi+ij\ 
< \ai+i^r+i+i\ < \bi+i,t+i\ for alH = 1, . . . ,fc - 1. 
Let n = max{|zi;i|, . . . , luit-il} and suppose that Wt = z satisfies 
one of the following: 

(1) l^fc.fel = n and z is an integer between ak+i^i and bk.i 

(2) la/c+i^r+fcl = n and z is an integer between ak,i and bk,i 

(3) |afc+i,r+fc| = n, 2; is an integer between 0^,1 and afc+1,1 and 
|afe+i,j| < |afe,i+i| for some 1 < i < fc - 1. 

Then set Cfc = bk,kak,k+r ■ ■ ■ ak,ibk,k-i ■ ■ ■ bk,i and define 



Dl = <Li... dk-iCk^k+i z wt+i ...w, 



5. Let k> 1 such that t = {k + l)(r + k + 2) < m and suppose 

W = €i... €k€k+l Z Wt+l ...Wn 

where each £j is a sequence of the form £j = Oj^j+r • • • cii^ibi^i . . . 
for 1 < i < A; and Cfe+i = ak+i,k+r+i ■ ■ ■ afe+i,i^fe+i,fe ■ • ■ Fur- 
ther suppose that the integers aij and bij, whenever they appear 
among Wi . . . Wt-i = £1 • • • £fc+i, satisfy the following conditions: 
ttij > and bij < (or vice versa) 
\aij-i\ < \aij\ < \ai+ij\ and < \bij \ < \bi+ij\ 

\ai,r+i\ < \bi,i\ < \ai+i^r+i+i\ for alH = 1, . . . , fc. 
Let n = max{|wi|, . . . , and suppose that Wt = z satisfies 

one of the following: 

(1) |afe+i^r+fe+i| = IT- and z is an integer between 0^+1^1 and bk+1^1 
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(2) |6A;+i,fc| = n and z is an integer between a^+i^i and bk,i 

(3) |&A;+i,fc| — n, z is an integer between bk^i and and 

< for some 1 < i < A: - 1. 

Then we set £fe£fc+i = a^^k+r ■ ■ ■ ak,iak+i,k+i+rbk,k ■ ■ ■ bk,iak+i,k+r 
■ ■ ■ ak+i,ibk+i,k ■ • • bk+1,1 and define 



, Cfc-iCfeCfe+izwt+i 



As promised, these generate the equivalence relations described above. 

Theorem 3.6 ([23 ). Two colored permutations w and v have the same left tableau 
in the image of the map Gr if and only if one can be obtained from the other via a 
sequence of operators for z = 1, 2, . . . , 5 

We proceed with a case by case examination of the action of each of the above 
generators on right tableaux and their effect on sign. Consider a signed permutation 
w € Hn and let Gr{w) — {P,Q) be its image among same-shape standard domino 
tableaux of rank r. Define 

F{w) = (-1)'' • (^-iypMP)+'^pMQ) . sign{P) ■ sign{Q). 

3.2.1. We first examine the operators D^. Note that D\{w) — siw for some i. 

Thus in all cases sign{D\{w)) — —sign{w). We verify that F{D'[{w)) = —F{w). 

Let Q' be the right tableau of Dl{w). There are two possibilities: 

1. Suppose that the action of on the right tableau of w ex- 
changes a block of dominos 



k 








I 


m 


with 


k 


I 








m 



or 



k 


1 




fe 






m 


with 




m 



while keeping the rest of the tableau fixed. For the sake of type- 
setting we are writing I — k -\- 1 and m = k + 2. Then this 
operation preserves the d statistic as well as spin. We examine 
sign{Q) = (— Recall that within domino tableaux, inver- 
sions are defined in terms of marked squares. First assume that in 
fact Dl changes 

into S' = 



k 


I 


m 



fe 


TTl 


i 



and that the top left-most box of k is marked. Then {k + 2,k + l) G 
Inv{Q') but not in Inv{Q). The only other changes in Inv{Q) 
occur when p is a marked square lying in the same rows as 5 in Q 
but outside of S. Then an inversion of the form {x,p) or {p,x) is 
exchanged for an inversion of the form (y,p) or (jp, y) where x,y d 
{k, k + 1, k + 2}. Consequently, sign{Q) ~ —sign{Q'). The other 
possibilities are similarly routine. 

2. If the action of is not by exchange of one of the above 
configurations, then by [7, 2.1.19] and [TSl Prop 4.6], Q and Q' 
differ by an exchange of labels of two consecutive dominos. It is 
clear that this changes inv{Q) by one. The other statistics are 
constant. 

In either case, F{Dl{w)) — —F{w), as desired. 



12 



THOMAS PIETRAHO 



3.2.2. The case of the operator D2 is very similar. Again, we have D2{w) — SiW for 
some i and consequently sign{D2{w)) — —sign{w). The description of the action 
of this operator on Q is implicit in the proof of |2Sl Theorem 3.1]. It exchanges 
two consecutive dominos. As above, this changes inv{Q) by one, holds the other 
statistics constant, and again, F{D'2{'w)) — —F{w), as desired. 



3.2.3. In the case of D^, we have D'!^{w) = tw. Consequently sign{D'^{w)) = 
— sign(w) . The action of on right tableaux is more intricate. Our description is 
based on [THl Cor 4.4 et scq.]. Within Q, the operator exchanges the subtableaux 



S 







































"3 1 




















Si 








82 













and S' 







































"2 1 










/3l 


















83 











a 



where the labels in S and S" coincide with Nr+2, cti = r + 2, and p may equal 
zero. The rest of Q is fixed. Note that the d statistic is fixed by this operation 
and spin{Q) changes by one. We examine sign{Q). First assume that r is odd and 
consequently the squares adjacent to the core are not marked squares. We have to 
consider the effect this transformation has on the set of inversions in Q. We consider 
two subsets, the inversions Inv{S) among entries in S, and those occurring between 
entries in S and the rest of Q. The order of the latter set is fixed by D3 as labels 
in S and S' are just We compare the size of the former set in S and S' . By 

inspection, 

Inv{S') \ Inv{S) = {(a„ l)}„,^i U {(ft, 1)},. 

Thus inv{S') = inv{S) + (r + 1). Since r is odd, this means sign(Q) = sign{Q'). 
Similar analysis applies in the case r is even. In either case, we have the equality 
F{D^{w)) — —F{w), as desired. 



3.2.4. At first glance, the operators D\ and Dl^ seem much more daunting than the 
prior three, but at least on the level of tableaux, they are in some sense just more 
intricate versions of D"!!^. We first note that by the construction of -D4, adopting 
notation from its definition, 

sign{Dl{w)) = {-lf+''+^sign{w). 

The effect of the operators D\ and Dl^ on right tableau is described in the proof of 
PSI Theorem 3.1]. There are four cases in our analysis of F{D\{w)) distinguished 
by the sign of the and the parity of r, which influences the choice of marked 
squares. First assuming that the aij are negative, the right tableaux Q of w and 
Q' of D\{w) differ within the subtableaux 
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and S' 



corresponding to the dominos inserted from the subword €k€k+i z. Here we define 
I = k + r + + t = {k + l){r + k + 1) , and adopt the shorthand Ij = I + j. 

Note that these tableaux are independent of the cases (l)-(3) in the definition of 
D^. The same subtableau results in all three. 

It is clear that d{Q) — d{Q') as this statistic only depends on the underlying 
tableau shape. Further, spin{Q) and spin{Q') differ by one. We analyze inv{Q). 
If A is a subtableau of B, we will write B \ A foi the boxes in B not included in 
A, and B{t) for the subtableau of B consisting of dominos with labels less than or 
equal to t. Note that 

inv{Q) = inv{S) + inv{Q{t) \ S) + inv{Q{t) \ S, S) 
+ inv{Q \ Q{t)) + inv{Q \ Q{t),Q{t)). 

The only values in this decomposition that can potentially change in the transfor- 
mation from Q to Q' are inv(S) and inv{Q{t) \ S,S). 

When r is odd, then the top rightmost squares of S and S" are unmarked and 
inv{S') - inv{S) = k - 1, while inv{Q'{t) \ S', S') - inv{Q{t) \S,S)= 0. When r 
is even, then then the top rightmost squares of S and S" are marked and inv{S') — 
inv{S) = k, while again inv{Q'{t) \ S', S') - inv{Q{t) \S,S)= 0. 

When the aij are positive, the tableaux Q and Q' differ in subtableaux that 
are transposes of S and S' . While the analysis is a little different, the above 
results are exactly the same: inv{Q') — inv{Q) = k — 1 when r is odd and k 
when r is even. Hence in all the cases, inv{Q') — inv{Q) = k + r mod 2 and 
sign{Q') = {—l)''~^'^sign{Q). Consequently, 

F{Dl{w)) = {-l)''+^+^F{w). 

and we find that F{w) transforms in step with sign{w), as desired. 

3.2.5. D^. Analysis of this operator follows a similar outline as that of Dl. We 
first note that by its construction, again adopting notation from the definition of 

sign{Dl{w)) = (-1)''+'^ sign{w) . 

Again there are four cases in our analysis of F{Dl{w)). First assuming that the Uij 
are positive, the right tableaux Q oiw and Q' of (w) differ within the subtableaux 
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corresponding to the doniinos inserted from the subword CfeCfc+i z. Here we define 
I = X]i=i(2* + ^ = + + fc + 2), and for integers j, Ij = Z + j, and 
Zj- = ^ + A; + r + j. Again, (i((5) = d{Q') and spin{Q) differs from spin{Q') by 1. 
We analyse inv{Q). 

When r is odd, then the top rightmost squares of S and S" are marked and 
inv{S') ~ inv{S) = fc + r + 1, and inv{Q'{t) \ S", S") - inv{Q{t) \ 5, 5) = 0. When 
r is even, then then the top rightmost squares of S and S" are unmarked and 
inviS') - inv{S) ^k + r, and again inv{Q'{t) \ S' , S') - inv(Q{t) \S,S)= 0. 

When the o^j are negative, the tableaux Q and Q' differ in subtableaux that 
are transposes of S and S' . We again have inv(Q') — inv{Q) = fc + r + 1 when 
r is odd and fc + r when r is even with the other inversion statistics unchanged. 
Hence in all the cases, inv{Q') — inv{Q) = fc mod 2 and sign{Q') = sign{Q). 
Consequently, 

The proof of Theorem |3.1| is complete. 
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